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Abstract. - We discuss a population of sequences subject to mutations and frequency- 
dependent selection, where the fitness of a sequence depends on the composition of the entire 
population. This type of dynamics is crucial to understand, for example, the coupled evolution 
of different strands in a viral population. Mathematically, it takes the form of a reaction- 
diffusion problem that is nonlinear in the population state. In our model system, the fitness is 
determined by a simple mathematical game, the hawk-dove game. The stationary population 
distribution is found to be a quasispecies with properties different from those which hold in 
fixed fitness landscapes. 



Introduction. - The roles of chance and determinism are a central theme in evolutionary 
biology. Eigcn's quasispecies theory ^ has been pivotal as a simple quantitative model for 
the intertwined effects of random mutations and Darwinian selection. These forces act on 
individuals with a genotype defined by an L- letter sequence a = (cti, . . . , ctl). The individuals 
reproduce at a rate /(ct) called the fitness of the genotype a, and are subject to random 
mutations of the sequence elements at a rate /i. The population is described by a time- 
dependent frequency distribution P{cr). The evolution of P{a) is described by a deterministic 
equation in the limit of large population size, when sampling errors in the reproduction- 
mutation process become negligible. For small fi, the evolution is dominated by reproductive 
success and produces a quasispecies, that is, a stationary population distribution ps(c) peaked 
around the genotype a* of maximal fitness. Large mutation rates, on the other hand, wipe 
out fitness differences and lead to a broad distribution. These two regimes are linked by a 
crossover which, depending on the fitness "landscape" f{a), may become a phase transition 
in the limit of infinite sequence length. The transition point is called the error threshold. 

Quasispecies evolution can be seen as a reaction-diffusion problem in sequence space that is 
conceptually related to problems in statistical physics. It takes the form of an (imaginary-time) 
Schrodinger equation for the population state, dtP = HP, with /(cr) as scalar potential and 
the kinetic term describing mutations ^ . This type of problem is mathematically tractable 
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since _ff is a linear operator, and has been studied for many different fitness landscapes. In 
particular, there can be extended subsets of sequence space — called neutral networks Fx — 
where the sequences encode the same phenotype x and, therefore, the fitness is constant. 
The evolution of the phenotype population p(x) = X^o-er ^i^^) '■^^^ often be described by a 
projected equation, dtp = H^p, taking into account the varying number of genotypes coding 
for the same phenotype. This means that the selection among different neutral networks is not 
only determined by their "fitness" (the reproduction rate) but also by their "robustness": i.e., 
to the probability that a random mutation leaves a genotype on the same neutral network [3| . 
In some cases, this effect can be described by a suitably defined "mutational entropy" 0]. 

The stationary distribution Ps(x) may again be a quasispecies around the phenotype x* 
of maximal fitness, while the sequence population within each neutral network Fx remains 
broad. Examples are RNA sequences cr with only the folding configuration x((t) determining 
the fitness. In this case, the neutral networks Fx consist of all sequences with the same fold 
xIH. 

A step towards a more realistic theory of mutation-selection processes is to take into 
account the dependence of the fitness on the population state. This is particularly apparent, 
for example, in viral evolution. The reproductive success of one viral strand will depend on 
the other strands that coexist in the same population. At the level of phenotypes, a well- 
known way to describe such coupled systems is evolutionary game theory i6||7||H||ni . Here we 
limit ourselves to the simplest form of a mathematical game, which is described by a set of 
basic strategies i = 1, . . . , s and a matrix A — (Aij) which specifies the relative fitness or 
payoff of strategy i played against strategy j. Then the phenotype x is a mixed strategy 
(xi, . . . ,Xs), where < < 1 is the probability that the individual plays basic strategy i. 
One can then define the average strategy x = {xi, . . . ,Xs) by averaging the phenotype over 
the whole population. A (simplified) model of the evolutionary process is called adaptive 
dynamics [SI ch. 9]. The time-dependent population state is approximated by a sequence of 
phenotypically homogeneous states, each evolving from the previous one by invasion of the 
neighboring mutant with the highest relative payoff. It can be shown that adaptive dynamics 
leads to strategic optimization: every stable fixed point x* of the population average is a Nash 
equilibrium that is, a mixed strategy that maximizes the payoff against itself (see the 
more precise definition below). 

In this Letter, we extend the quasispecies approach to populations evolving according 
to evolutionary games. This is appropriate for the example of viral evolution, where the 
frequency-dependent fitness differences are balanced by mutations (occuring with a high rate 
of 10"'' per nucleotide per generation). Unlike for standard quasispecies theory, the "Hamil- 
tonian" H is now a nonlinear operator acting on the population state. Quite remarkably, the 
stationary population distributions Ps(c) and Ps(x) can still be calculated exactly in some 
cases. We will demonstrate this for sequences playing the hawk-dove game, one of the sim- 
plest games with a nontrivial Nash equilibrium. An individual's phenotype is associated with 
a mixed strategy x uniquely determined by its genotype a. There are extended neutral net- 
works Fx consisting of all sequences encoding the same strategy x. The phenotype population 
p{x) is found to be a quasispecies. The population average x deviates from the Nash equilib- 
rium by an amount depending on the mutation rate. The properties of the quasispecies are 
different from those in a fixed fitness landscape. This reflects the fact that the system exhibits 
a higher degree of near-neutrality, as will be discussed in detail below. 

Phenotypes and evolutionary game theory. - Consider a population whose phenotypes 
correspond to the mixed strategies x of a game with payoff matrix A. That is, an individual 
of phenotype x plays the basic strategy i with probability Xi (i = 1, . . . , s), and the payoff for 
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a mixed strategy x against a mixed strategy x' is assumed to be bilinear: 

s 

(^(x|x') = ^ AijX^x'j . (1) 

A Nash equilibrium is defined to be a strategy x* that is optimal against itself, i.e., 

0(x|x*) < 0(x*|x*) for all strategies x . (2) 

In evolutionary game theory, the game payoff determines the relative fitness of individuals. 
The population state is a time-dependent phenotype distribution p(x); we suppress the de- 
pendence on t in the notation here and below. Assuming random mixing of the population, 
the fitness of a phenotype x is given by 

ydx>(x|x')p(x') = 0(x|x), (3) 

where x denotes the average strategy. Hence, the population evolves according to 

9tp(x) = [0(x|x)-0(x|x)]p(x) ; (4) 

subtracting the average fitness (/)(x|x) ensures that the normalization of the distribution p(x) 
is preserved. In general, this equation does not have a unique stationary solution. It is 
easy to see that p(x) is an attractive fixed point if and only if the population average x is 
a Nash equilibrium and all phenotypes x in the support of p are degenerate in fitness, i.e., 
<;6(x|x*) =^(x*|x*). 

In this letter, we will focus on the classical hawk-dove game, which has the two basic 
strategies "hawk" {i = 1) and "dove" {i = 2); see, e.g., chap. 6]. Consider individuals of a 
population competing for resources of reproductive value A. Doves avoid confrontation, while 
hawks escalate fights. Thus, on average, two doves will share the resources. A dove meeting 
a hawk will leave the entire resources to the hawk. Two hawks will also share the resources 
but have to pay a confrontation cost CA. These payoffs can be written in matrix form, 

A-("-„'^"^;0^ (5) 

An individual with strategy x plays, by definition, hawk with probability xi = x and dove 
with probability X2 = 1 — x. The unique Nash equilibrium of the hawk-dove game is x* = 
min(l,l/C). In the following, we assume C > 1 so that the Nash equilibrium is a mixed 
strategy. According to eqs. JSJ and (0), the relative fitness of an arbitrary mixed strategy is 
given by 

(t){yi\±) - (t){yi\yi) = —{x - x){x* - x) . (6) 

Genotypes and mutations. - The phenotype of an individual is assumed to be uniquely 
determined by its genotype. We consider here genotypes with L loci contributing additively 
to the phenotype. Each locus has c alleles, of which a encode the hawk strategy and c — a 
the dove strategy. We denote hy q = a/c the relative fraction of hawk coding alleles. Such 
genotypes can be represented by sequences a = (cti, . . . ,(7l) whose letters take the values 
and 1 representing dove and hawk alleles, respectively. The corresponding phenotype is given 

by 

1 ^ 
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and takes the discrete values x = k/L, where k — 1,.. . ,L is the number of hawk alleles. 
During a time interval of duration dt any given locus in an individual's genotype mutates 
with probability /i dt into a randomly chosen allele. It is easy to see that in the absence of 
selection the mutations change the discrete population distribution, 

dtp{x) = -[J{x)-J{x-L-')], (8) 

where J{x) is the net probability current between all genotypes with k = Lx and with fc + 1 
hawk alleles: 

Ji^) - - x)qp{x) -ix + - q)p{x + L-')] . (9) 

This neutral evolution leads to a stationary state po{x) where all alleles are equally probable: 

Po(a:) = . (10) 



\Lx 

Hence the average phenotype in the population is xq = q. 

Quasispecies equation. ~ In the following, we describe systematically the interplay be- 
tween mutations and selection, which leads to nontrivial stationary population states Psix). 
Assuming that the two kinds of processes act in parallel, we obtain the mutation-selection 
equation 

dtp{x) = - [J(x) - J{x - 1/L)] + ^{x - x){x* - x)p{x) . (11) 

Zx 

This type of dynamics is usually referred to as paramuse models The L dependence in the 
fitness has been introduced in order to obtain a well-behaved large-L limit, i.e., a distribution 
of the form p{x) oc exp (— LjF(a;)), in analogy with the thermodynamic limit in statistical 
mechanics and with the standard quasispecies theory Since the phenotype average x 
depends on p(a;), eq. 1)11(1 leads to a nonlinear equation for the stationary population state 

Ps{x). 

In the present case, the stationary solution can be obtained exactly. Indeed, one can 
argue that each different locus undergoes an independent mutation-selection process, where 
the only interaction is encoded in x. Thus one may look for a probability distribution that 
factorizes into a product of single-locus probabilities (this is a general property when the 
fitness is a linear functional of the genome |llj). Since all loci are equal in the present model, 
the stationary distribution Psix) in the presence of selection is still a binomial, but with a 
different average x: 



The scaled variance of this distribution, = L (x ~ x)^ , is linked to to x by 

a^ = x{l-x). (13) 

On the other hand, the equation of motion implies a hierarchy of evolution equations for 
the moments of p{x). The leading equation of this hierarchy reads 

dtx^^(x-q) + ^a'{x~x*). (14) 

Inserting eq. we obtain a self-consistency equation for x, which is of cubic order. The 
resulting values of x and at stationarity are plotted in Fig. 1 against the effective mutation 
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rate /Lt/A together with the results from numerical simulations for L = 16. In accordance with 
the results obtained above, there is no L-dependence in these quantities (except for the effects 
of rounding errors). 




Fig. 1 - Average strategy x (left) and strategy variance (right) of the phenotypic quasispecies as 
a function of the effective mutation rate /^/A. The points are simulation data for c = 4, a = 1, A = 1, 
X* = 0.2, L — 16. The continuous lines give the solutions of eqs. H1.'-{I14|I . The dashed lines represent 
the solution obtained from the fi-expansion by neglecting the last term in eq. HSU . 

Note that a straightforward application of the well-known il-cxpansion (see, e.g., [121 
Chap. IX]) overestimates the variance of x and therefore does not give the correct result. This 
can be seen from the next-to-leading evolution equation for the moments oi p{x), 

2 _ lic 



dta' = ^ \-2a' + x(l - 2q) + q\ + —L\x* - x){x - xf. (15) 

c — 1 '■ ■' 2x* 

In the ri-expansion it is assumed that the first two moments can be obtained self-consistently 
from the truncated system of equations H14(l and (|15|l . neglecting all higher cumulants as 
L — > 00. Indeed, the m-th order cumulant of Ps{x) scales like L^~™ as required by the 
fi-expansion. However, in eq. p5|l the skewness is multiplied by a factor L^, so that its 
contribution to remains finite in the limit L —> oo and the truncation fails. 

As a function of /i/A, the solution Ps{x) describes a crossover between two dynamical 
regimes. 

(i) In the fast mutation regime fj,/X 3> 1, the evolution becomes effectively neutral. The 
phenotype distribution approaches the asymptotic form po{x) given by eq. we have 

x^q + o(^^^ . (16) 

(ii) In the slow mutation regime fi/X <S 1, the phenotype average approaches the Nash equi- 
librium but the variance remains finite, 

x = x* + ^^iq-x*) + o[t£l , a'=x*il-x*) + 0[t£j . (17) 

The salient feature of this crossover is that the phenotypic quasispecies is always broad, 
even in the limit /i — > 0. (In contrast, the standard quasispecies in a fixed fitness landscape 
would be localized around a master phenotype with a variance = 0{ii).) This broadness 
reflects the fact that at a mixed Nash equilibrium, all participating basic strategies have the 
same fitness. Hence, as x approaches x* , the fitness differences between phenotypes become 
small. 
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Discussion. - A generic feature of this mutation-selection dynamics is broad equilibria 
around Nash points, with a larger degree of sequence divergence than in many fixed fitness 
landscapes. It is clear that this method can be applied without major changes to evolutionary 
games with unconditional strategies, in which the payoff 0(x|x') is a linear function of x'. 
The expression of the mutational entropy can be easily generalized to games with s pure 
strategies, encoded hy ai (i = 1, s) alleles respectively. Systems where different loci have 
a different weight in the genotype-phenotype mapping |(7J) can be treated along the lines of 
ref. 0]. In general, the stationary distribution of strategies will be close to an evolutionary 
stable strategy for small mutation rate, if there is one. The situation is different in games 
with conditional strategies, like the celebrated Prisoner's Dilemma game |S1 p. 101]. In 
this case, the payoff of a given strategy depends on more properties of the population than 
the average strategy alone. This case will be the subject of a separate publication ^1]. It 
is also clear that the present approach can be generalized to the case of asymmetric games, 
which can be used to model the co-evolution of different (but interacting) populations. 

The main limitation of the quasispecies approach is the assumption of large population 
sizes. In finite populations, there are sampling fluctuations, which lie at the heart of the 
Neutral Evolution approach |15[ll6j . Recently these fluctuations have been incorporated into 
phenotypical evolutionary game theory, using a quantum- mechanical formalism |17j . Anal- 
ogous finite-population effects in our co-evolutionary sequence dynamics lead to a quantum 
field theory, which will also be the subject of future work 14 . 

* * * 
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